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In  order  for  an  Intelligent  Tutoring  System  (ITS)  to  correct  students’  exercises,  it  must  know  how  to  solve 
the  same  type  of  problems  that  students  do  and  the  related  knowledge  components.  It  can,  thereby,  com¬ 
pare  the  desirable  solution  with  the  student’s  answer.  This  task  can  be  accomplished  by  an  expert  system. 
However,  it  has  some  drawbacks,  such  as  an  exponential  complexity  time,  which  impairs  the  desirable 
real-time  response.  In  this  paper  we  describe  the  expert  system  (ES)  module  of  an  Algebra  ITS,  called 
PAT2Math.  The  ES  is  responsible  for  correcting  student  steps  and  modeling  student  knowledge  compo¬ 
nents  during  equations  problem  solving.  Another  important  function  of  this  module  is  to  demonstrate 
to  students  how  to  solve  a  problem.  In  this  paper,  we  focus  mainly  on  the  implementation  of  this  module 
as  a  rule-based  expert  system.  We  also  describe  how  we  reduced  the  complexity  of  this  module  from 
0(nd)  to  0(d),  where  n  is  the  number  of  rules  in  the  knowledge  base,  by  implementing  some  meta-rules 
that  aim  at  inferring  the  operations  students  applied  in  order  to  produce  a  step.  We  evaluated  our 
approach  through  a  user  study  with  forty-three  seventh  grade  students.  The  students  who  interacted 
with  our  tool  showed  statistically  higher  scores  on  equation  solving  tests,  after  solving  algebra  exercises 
with  PAT2Math  during  an  approximately  two-hour  session,  than  students  who  solved  the  same  exercises 
using  only  paper  and  pencil. 
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1.  Introduction 

Intelligent  Tutoring  Systems  (ITSs)  have  shown  promising  results 
when  applied  as  a  supplemental  classroom  learning  tool  (Koedinger, 
Anderson,  Hadley,  &  Mark,  1997;  Nicaud,  Bittar,  Chaachoua,  Inam- 
dar,  &  Maffei,  2006;  Nicaud,  Bouhineau,  &  Huguet,  2002).  Large- 
scale  experiments  in  high-schools  demonstrated  that  ITSs  can  im¬ 
prove  students  learning  (Koedinger  et  al„  1997;  Koedinger  &  Sueker, 
1996).  The  success  of  this  type  of  educational  software  is  due  to  the 
fact  that  it  can  offer  important  features  to  personalize  the  learning 
processes  such  as  one-on-one  learning,  immediate  personal  feed¬ 
back,  demonstration  of  problem  solving  when  students  are  having 
difficulty,  and  assessment  of  students’  skills. 

Vanlehn  (2006)  describes  the  tutor  as  having  two  loops.  The 
outer  loop  is  responsible  for  deciding  the  sequence  of  exercises  or 
problems  for  students  to  work  on.  The  inner  loop  provides  step- 
by-step  guidance  during  problem  solving  activity. 
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In  order  to  provide  immediate  feedback  in  the  inner  loop,  an 
ITS’  architecture  is  generally  composed  of  an  expert  system 
module  (ES)  that  is  able  to  solve  the  same  type  of  exercises  that 
students  should  do  in  multiple  ways.  Thus,  for  each  step  of  the 
problem  the  system  compares  the  answer  provided  by  a  student 
with  the  expert  system's  solutions  (an  answer  may  have  several 
correct  solutions)  and  checks  whether  they  are  equivalent.  If  the 
ES  can  generate  or  test  all  possible  solutions  to  a  given  prob¬ 
lem,  then  it  can  identify  when  a  student  has  taken  an  incorrect 
path  to  solve  the  given  problem  and  offer  immediate  feedback 
(generally  colored  labels  are  presented  to  indicate  whether  or 
not  the  student’s  answer  for  a  given  step  is  correct)  (Heffernan, 
Koedinger,  &  Razzaq,  2008).  Some  tutors  provide  additional  re¬ 
sources  (e.g.  explanations  or  hints)  when  students  are  having 
difficulty  or  solving  the  problem  or  arriving  at  the  correct 
answer. 

In  fields  such  as  math  and  physics,  the  knowledge  is  usually 
implemented  as  a  rule  in  the  form:  "if  ( condition  is  true )  then  (do 
action  A)”.  Each  rule  represents  an  operation  that  can  be  applied 
in  a  step  to  solve  a  problem.  The  ES  inference  engine  scans  the  base 
searching  for  rules  to  be  triggered,  i.e.  rules  whose  conditions  are 
satisfied  by  the  current  step  of  the  solution. 
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In  addition  to  providing  appropriate  feedback  to  students,  the 
rules  also  contain  information  about  the  related  knowledge  com¬ 
ponents.  For  example,  a  rule  can  represent  the  process  (knowledge 
component  or  skill)  of  “subtracting  an  integer  b  on  both  sides  of 
the  equation”.  Thus,  when  the  ES  module  corrects  a  student  step 
solution,  it  is  able  to  provide  the  student  model  with  information 
about  the  skills  necessary  to  solve  that  step.  The  student  model 
uses  this  information  to  infer  which  skills  students  have  mastered 
and  which  they  need  to  practice  more.  This  allows  the  tutor  to  cre¬ 
ate  personalized  hints  in  the  inner  loop  and  also  to  select  more 
appropriate  exercises  for  students  to  solve  (outer  loop). 

Although  there  are  well  known  algebra  tutors  (Chaachoua,  Ni- 
caud,  Bronner,  &  Bouhineau,  2004;  Cohen,  Beal,  &  Adams,  2008; 
Koedinger  &  Sueker,  1996;  Melis,  Goguadze,  Libbrecht,  &  Ullrich, 
2009),  few  of  them  have  an  expert  system  module  that  is  able  to 
solve  exercises  and  provide  step-by-step  guidance  (Chaachoua 
et  al.,  2004;  Koedinger  &  Sueker,  1996),  an  essential  feature  for  a 
learning  system  to  be  classified  as  an  ITS,  according  to  (Vanlehn, 
2006).  Furthermore,  previous  work  does  not  explore  in  detail 
how  to  implement  an  expert  system  module,  which  artificial  intel¬ 
ligence  knowledge  representation  format  needs  to  be  used,  when 
an  inference  mechanism  should  be  trigged  and  how  to  solve  some 
inherent  computational  complexity  problems. 

This  paper  presents  the  ES  of  the  algebra  tutor  PAT2Math.  PAT2- 
Math  is  an  intelligent  tutor  system  that  teaches  students  how  to 
solve  linear  and  quadratic  equations.  It  is  a  web  system  imple¬ 
mented  in  Java,  which  allows  students  to  use  it  in  any  computer 
or  platform  with  Internet  access.  PAT2Math  is  composed  of  an 
algebra  editor  (PATequation),  which  assists  students  in  solving 
equations. 

The  ES  has  an  essential  role  in  PATequation;  it  is  responsible  for 
providing  immediate  feedback  to  students  at  every  step  of  their 
problem  solving.  Our  main  goal  is  to  present  this  module  knowl¬ 
edge  and  explain  how  to  the  ES  implements  problem  solving  and 
provides  the  student  with  step-by-step  guidance.  We  describe 
how  to  reduce  the  complexity  of  this  module  from  0(nd)  to  O(d), 
where  n  represents  the  number  of  rules  in  the  knowledge  base, 
by  using  meta-rules  that  guide  the  inference  of  the  operations  stu¬ 
dent  applied  to  produce  a  step.  We  finish  this  paper  by  presenting 
the  results  of  a  user  study  we  conducted  with  forty-three  7th  grade 
students  who  interacted  with  PATequation  for  three  classes. 

This  paper  is  organized  as  follows.  Section  2  describes  problem 
solving  under  a  pedagogical  perspective.  Section  3  presents  the 
current  state  of  the  art  in  Algebra  Intelligent  Tutoring  Systems.  In 
Section  4,  we  explain  the  main  artificial  intelligence  techniques 
used  to  develop  ITS  expert  systems.  In  Section  5,  we  describe  PAT2- 
Math,  the  Algebra  Tutor  that  our  research  group  is  developing. 
PATequation,  the  problem  solving  editor  of  PAT2Math,  is  presented 
in  Section  6.  The  ES  responsible  for  providing  step-by-step  guid¬ 
ance  in  PATequation  is  described  in  Section  7.  The  experiment  de¬ 
sign  and  results  are  reported  in  Section  8.  Finally,  Section  9 
presents  our  conclusions. 


2.  Solving  algebraic  problems 

An  Algebra  task  is  generally  a  word  problem  for  the  student  to 
solve.  An  algebraic  word  problem  consists  of  one  or  more  sen¬ 
tences  representing  a  situation  or  a  story,  where  the  student  needs 
to  understand  the  elements  in  order  to  generate  a  mathematical 
model  to  represent  it.  The  model  consists  of  one  or  more  equations 
that  the  pupil  should  solve  in  order  to  obtain  the  numerical  values 
that  are  the  solution  of  the  problem  (Gama,  2004).  Take  for  exam¬ 
ple  the  word  problem  below  (adapted  from  Munem  &  West  (2003, 
p.  107)): 


“A  computer  store  sells  desktop  and  laptop  computers.  Due  to  space 
considerations,  the  number  of  laptops  in  inventory  is  seven  less 
than  twice  the  number  of  desktops  in  stock.  How  many  desktops 
does  the  store  have  if  it  has  a  total  of 272  computers?" 


The  process  of  solving  a  word  problem  has  two  phases  (Mayer, 
1999;  Polya,  2004):  (i)  the  Problem  Representation  (also  called 
Symbolization  (Heffernan,  Koedinger,  &  Razzaq,  2008)),  and  (ii) 
the  Problem  Solution.  While  the  former  concerns  to  the  transfor¬ 
mation  of  algebra  word  problems  into  a  system  of  equations,  the 
second  encompasses  the  process  of  solving  these  equations  using 
algebraic  operations. 

For  example,  for  the  word  problem  that  we  previously  pre¬ 
sented,  the  student  could  provide  the  following  solution: 


X  +  (2x  —  7)  =  272  (1) 

3x  -  7  =  272  (2) 

3x  =  279  (3) 

x=93  (4) 


In  the  example  above,  line  (1 )  refers  to  the  process  of  Problem  Rep¬ 
resentation,  and  lines  (2-4)  to  the  Problem  Solution  phase.  As 
shown  in  the  above  solution,  solving  a  task  involves  several  steps. 
Each  line  provided  by  the  student  in  the  above  solution  is  a  step. 

A  step  can  involve  the  correct  use  of  one  or  more  Knowledge 
Components  (KC)  (also  called  knowledge  units  (Aleven,  McLaren, 
Sewall,  &  Koedinger,  2009)).  It  comprises  any  unit  into  which  the 
knowledge  can  be  broken  down,  such  as  rules,  concepts,  facts, 
and  procedures  (Vanlehn,  2006).  For  instance,  in  order  to  arrive 
at  line  (2)  in  the  above  example,  the  student  applied  the  operation 
(or  KC)  “add  variable  coefficients"  in  line  (1)  of  the  equation. 

In  the  next  section,  we  will  describe  the  main  Algebra  Intelli¬ 
gent  Tutoring  Systems  and  the  tools  and  types  of  feedback  they  of¬ 
fer  to  help  students  solve  algebra  word  problems  in  these  two 
phases. 

3.  Algebra  Intelligent  Tutoring  Systems 

The  field  of  ITS  has  shown  significantly  improvements  since  the 
emergence  of  the  first  systems  in  the  eighties  (Woolf,  2009).  The 
evolution  of  the  Internet,  the  increasing  performance  of  computers, 
and  improvements  of  artificial  intelligence  techniques  and  tools 
have  furthered  the  development  of  ITSs  in  several  domains,  such 
as  Physics,  Math,  Medicine  and  others  (see  (Woolf,  2009)  for  an 
overview). 

Previous  work  has  largely  been  applied  in  classroom  settings, 
demonstrating  they  can  improve  student  performance  on  stan¬ 
dardized  and  experimenter-designed  tests  by  one-half  to  two  stan¬ 
dards  deviation  (Cohen  et  al„  2008;  Koedinger  &  Sueker,  1996; 
Shelby  et  al.,  2000).  Some  of  the  most  known  research  focused 
on  the  Algebra  content  domain.  This  is  the  case  of  Cognitive  Alge¬ 
bra  Tutor  (previously  PAT)  (Koedinger  &  Sueker,  1996),  Aplusix 
(Nicaud  et  al„  2006,  Nicaud,  Bouhineau,  &  Huguet,  2002),  Active- 
Math  (Goguadze  &  Melis,  2008;  Melis,  Goguadze,  Libbrecht,  &  Ull¬ 
rich,  2009)  and  AnimalWatch  (Birch  &  Beal,  2008;  Cohen  et  al., 
2008).  We  believe  there  are  two  main  reasons  for  this.  First,  Alge¬ 
bra  is  a  content  domain  (or  task  domain  (Vanlehn,  2006))  in  which 
a  great  number  of  students  experience  poor  achievement  (Carpen¬ 
ter,  Kepner,  Corbitt,  Lindquist,  &  Reys,  1982;  National  Commission 
on  Excellence  in  Education,  1983).  Secondly,  it  requires  less  effort 
to  formalize  math  content  into  computer  algorithms,  because  it 
is  mainly  composed  of  procedural  content,  which  can  be  easily  rep¬ 
resented  by  computer  algorithms.  In  the  end  of  this  section,  we  de¬ 
scribe  the  main  algebraic  tutors  proposed  by  the  Artificial 
Intelligence  and  Education  community. 
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AnimalWatch  (Birch  &  Beal,  2008;  Cohen  et  al.,  2008)  assists 
middle  school  students  in  solving  arithmetic  word  problems.  The 
mathematical  problems  explore  environmental  issues  around 
endangered  animal  species.  The  main  goal  is  to  motivate  students 
by  connecting  math  to  real  problems  involving  animals.  The  main 
screen  of  AnimalWatch  is  composed  of  a  textual  description  of  a 
word  problem,  additional  illustrations  of  the  problem  (such  as  vi¬ 
deo,  charts,  diagrams  and  pictures)  and  an  answer  box.  The  student 
solves  the  problem  elsewhere  (for  instance,  on  paper)  and  provides 
the  final  result  in  the  answer  box  (one  step  in  the  tutor  interface). 
AnimalWatch  is  mainly  concerned  with  the  Problem  Representa¬ 
tion.  Thus,  it  does  not  have  an  ES  that  is  able  to  correct  the  problem 
in  an  intermediary  step  of  the  solving  process.  If  student  needs 
help,  the  tutor  provides  hints,  but  it  is  not  able  to  identify  which 
steps  of  the  student’s  problem  solving  are  incorrect  if  the  student’s 
answer  does  not  match  the  expected  result. 

ActiveMath  (Goguadze  &  Melis,  2008;  Melis  et  al.,  2009)  is  a 
web-based  intelligent  learning  system  for  math  that,  in  addition 
to  algebra,  teaches  differential  calculus,  logics,  statistics.  It  allows 
the  students  to  freely  navigate  around  the  content.  It  uses  a  seman¬ 
tic  web  representation  in  order  to  choose  learning  objects  for  stu¬ 
dents,  which  can  be  textual  explanations,  several  types  of  exercises 
(multiple-choice,  word  problem),  and  others.  Like  AnimalWatch, 
ActiveMath  is  more  concerned  with  the  outer  loop,  i.e.  using  a  stu¬ 
dent  model  and  web  semantic  information  to  decide  which  task  is 
presented  to  the  student.  It  does  not  incorporate  an  ES  that  sup¬ 
ports  students  in  the  inner  loop. 

Aplusix  and  Cognitive  Tutor  do  include  an  ES  that  is  responsible 
for  providing  step-by-step  guidance  to  the  student  during  a  task. 
During  the  inner  loop,  a  tutor  can  offer  customized  mentoring  in 
one  or  both  of  the  problem  solving  processes  explained  in  Section  2 
(i.e.  Symbolization  or  Problem  Solution).  Cognitive  Tutor  gives  the 
student  a  linear  or  quadratic  word  problem  to  solve  with  the  help 
of  charts  and  a  table  that  assist  students  in  the  Symbolization.  It 
also  has  a  simple  editor  for  students  to  solve  the  linear  equation. 

On  the  other  hand,  the  main  interface  of  Aplusix  is  an  editor 
where  students  can  solve  math  problems  in  the  fields  of  numerical 
calculations,  expansions,  factorizations,  quadratic  equations, 
inequalities,  and  systems  of  equations.  It  is  not  an  ITS  in  the  sense 
that  it  does  not  retain  information  about  the  skills  that  a  student 
mastered  and  it  does  not  choose  the  next  task  based  on  a  student 
model  (outer  loop).  For  each  step  provided  by  the  student,  Aplusix 
is  able  to  provide  real  time  feedback  by  showing  whether  or  not 
the  answer  is  correct  or  by  demonstrating  how  to  solve  the 
equation. 

Both  expert  systems  (Aplusix  and  Cognitive  Tutor  (CT))  were 
implemented  as  rule-based  expert  systems.  This  is  also  the  case 
of  the  proposed  expert  system.  The  main  differences  among  these 
works  are  the  information  associated  with  the  rules  and  the  inter¬ 
face  options,  which  enable  (or  not),  more  powerful  modes  of  inter¬ 
action  and  feedback  for  the  student.  The  ES  of  the  Cognitive  Tutor 
is  also  called  Cognitive  Model,  since  it  is  based  on  a  cognitive  ap¬ 
proach  to  learning.  It  contains  rules  that  represent  the  procedural 
knowledge  of  the  student  (Anderson,  Corbett,  Koedinger,  &  Pelle¬ 
tier,  1995;  Ritter,  Anderson,  Koedinger,  Si  Corbett,  2007).  Generally, 
each  rule  symbolizes  an  operation  (a  knowledge  component)  nec¬ 
essary  to  solve  an  equation.  However,  the  interface  of  the  solver 
limit  what  students  can  do  by  making  them  choose  one  operation 
from  a  menu.  Afterwards,  students  simply  have  to  complete  the 
new  textboxes  presented  by  the  system.  It  does  not  allow  the  stu¬ 
dents  to  apply  their  own  paper-and-pencil  reasoning  steps  and 
strategies  during  the  problem  solving.  One  limitation  of  Aplusix 
is  that  it  does  not  assess  student  knowledge.  Table  1  summarizes 
the  inner  loop’s  main  problem-solving  functionalities  supported 
by  Aplusix  and  Cognitive  Tutor.  It  is  important  to  observe  that 
the  focus  of  this  paper  is  on  problem  solving.  For  this  reason,  we 


Inner  loop  functionalities  supported  by  Algebra  ITSs. 

Supported  functionalities  Cognitive  Aplusix  Pat2Math 

Minimal  feedback  on  steps  x 

Error-specific  feedback 
Hints  on  the  next  step 
End-of-problem  review  of  the 
solution 

Assessment  of  knowledge  x 


are  not  taking  into  account  functionalities  provided  in  the  Symbol¬ 
ization  process. 

The  three  ITSs  provide  minimal  feedback  on  steps,  i.e.,  they  pro¬ 
vide  real-time  correction  in  the  form  of  a  symbol  (label,  color  or 
sign)  that  indicates  whether  or  not  a  student  response  is  correct. 
If  the  student  did  not  correctly  solve  a  step,  the  system  should  pro¬ 
vide  some  specific  feedback  indicating  how  to  solve  the  step.  This 
is  done  by  the  Hint  module  of  Pat2Math  and  by  the  companion 
agent  in  Aplusix.  Although  Cognitive  Tutor  presents  this  option 
for  the  Symbolization  process,  it  does  not  work  in  the  solver  tool. 
Another  function  available  in  CT  and  Pat2Math  is  the  assessment 
of  student  knowledge.  In  the  CT,  a  Bayesian  network  assesses  a  stu¬ 
dent’s  skills  and  displays  them  in  the  “skillometer”.  Although  Pat2- 
Math  is  able  to  track  a  student’s  performance  in  each  knowledge 
component  (currently  using  the  arithmetic  mean),  it  exhibits  the 
student’s  general  performance  in  a  colored  bar. 

In  the  next  section,  we  describe,  in  more  detail,  the  main  artifi¬ 
cial  intelligence  techniques  used  in  the  implementation  of  ES  in 
Algebra  tutors. 

4.  Implementing  the  expert  system  of  an  algebra  tutor 

Artificial  Intelligence  classifies  knowledge  into  two  types: 
declarative  and  procedural  (Nilsson,  1982).  Procedural  knowledge 
is  concerned  with  understanding  how  to  do  things,  such  as,  when 
to  apply  the  distributive  operation  in  a  quadratic  equation.  Declar¬ 
ative  knowledge  is  concerned  with  facts  -  for  example,  knowing 
that  the  capital  of  Brazil  is  Brasilia.  Generally,  the  Domain  Knowl¬ 
edge  module  of  an  ITS  is  able  to  represent  both  of  them. 

These  two  types  of  knowledge  usually  require  different  forms  of 
representation  (Davis,  Shrobe,  &  Szolovits,  1993).  A  usual  represen¬ 
tation  format  for  procedural  knowledge  is  a  production  rule.  A  rule 
is  typically  expressed  as  IF-THEN  statements  that  denote:  (1)  the 
antecedent  (IF)  that  embodies  a  set  of  conditions  to  be  satisfied; 
(2)  the  consequent  (THEN)  that  contains  actions  to  be  executed 
or  new  knowledge  to  be  produced  if  the  consequent  is  true.  An 
example  of  a  rule  is: 

IF  the  temperature  is  below  0 

THEN  the  weather  is  cold 

Rules  are  usually  used  by  expert  systems  to  represent  proce¬ 
dural  knowledge  and  allow  powerful  reasoning  and  inference.  Be¬ 
sides  having  a  knowledge  base  (KB)  that  stores  permanent  rules 
and  facts,  a  Rule-Based  Expert  Systems  (RBESs)  also  has  a  Working 
Memory  that  stores  the  facts  and  rules  that  are  being  applied  in  the 
current  problem. 

However,  the  most  important  component  of  a  RBES  is  the  infer¬ 
ence  engine.  It  selects  those  rules  whose  antecedents  match  Work 
Memory  facts  and  stores  them  in  the  Conflict  Set  (also  called  the 
Agenda).  If  the  Conflict  Set  has  more  than  one  rule  (i.e.,  if  more 
than  one  rule  can  be  triggered),  the  inference  engine  uses  a  conflict 
resolution  strategy  to  determine  which  rule  to  trigger.  A  RBES  is  a 
powerful  reasoning  mechanism  when  there  is  not  a  pre-specified 
sequence  of  actions  to  solve  the  problem,  such  as  in  an  exhaustive 
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search  or  in  the  case  of  Algebra,  where  students  can  solve  a  prob¬ 
lem  in  several  ways. 

The  inference  engine  is  the  most  important  component  of  a 
RBES  and  the  most  difficult  one  to  implement.  On  the  other  hand, 
it  is  independent  of  the  domain.  Although  the  facts  and  rules  are 
exclusive  for  each  domain,  the  inference  engine  depends  only  on 
the  language  used  for  knowledge  representation.  For  this  reason, 
there  are  some  implementations  of  the  inference  engine  that  assist 
the  development  of  a  RBES.  They  are  called  Expert  Systems  Shells. 
Drools  is  an  ES  shell  implemented  in  Java  (Bali,  2009;  JBOSS,  n.d.). 

The  PATEquation  expert  system  is  a  production  system  imple¬ 
mented  in  Drools,  version  4.0  (JBOSS,  n.d.).  We  chose  the  shell 
Drools  because  it  allows  easy  integration  with  Java.  It  is  possible 
to  invoke  Java  methods  from  Drools  and  vice  versa.  This  enables 
us  to  develop  more  powerful  and  flexible  rules. 

In  the  next  sections  we  will  present  more  details  about 
Pat2Math. 

5.  PAT2Math 

In  this  section,  Pwe  present  the  ITS  PAT2Math  and  the  function¬ 
alities  it  offers  to  assist  students  in  learning  algebra. 

PAT2Math  stands  for  Personal  Affective  Tutor  to  Math.  The  main 
goal  of  this  project  is  to  develop  a  web-based  ITS  that  takes  into  ac¬ 
count  students  emotions  (inferred  by  student’s  actions  in  the  sys¬ 
tem  interface  and  by  facial  expression  (Jaques,  Vicari,  Pesty,  & 
Martin,  2011))  when  scaffolding,  and  information  about  skills  they 
mastered,  as  an  ITS  usually  does.  The  domain  of  Algebra,  more  spe¬ 
cifically  the  content  of  linear  and  quadratic  equations,  was  chosen 
because  it  usually  induces  fear  and  other  negatives  emotions  in 
students  that  generally  impair  the  learning  process  and  result  in 
frustration  and  increased  dropout  rates  (Frenzel,  Pekrun,  &  Goetz, 
2007;  Hannula,  2002). 

Although,  there  are  well-known  ITS  in  literature  (cited  on  Sec¬ 
tion  3),  we  chose  to  develop  another  Algebra  ITS  because  most  of 
the  existing  systems  (such  as  Aplusix  and  Cognitive  Tutor)  are 
not  open-source.  We  cannot  extend  or  improve  on  them  to  inte¬ 
grate  the  emotions  inference.  Our  intention  is  to  make  PAT2Math 
available  without  charge  to  all  schools  in  Brazil.  Our  second  goal 
is  to  develop  an  ITS  that  follows  the  algebra  curriculum  specified 
by  the  Brazilian  Ministry  of  Education  (Brasil.  Secretaria  de  Edu- 
ca^ao  Fundamental,  1997).  Since  all  Brazilian  schools  must  follow 


this  specification  curriculum,  this  would  increase  the  acceptance 
of  our  ITS  in  Brazil. 

The  project  began  on  2008  and  we  have  since  developed  PAT- 
equation,  a  tool  that  includes  an  editor  and  a  powerful  expert  sys¬ 
tem  that  assists  students  in  solving  first  and  second  degree 
equations.  This  tool  comprises  the  inner  loop  of  PAT2Math  for 
problem  solving  and  offers  the  services  detailed  in  Table  1. 

Due  to  space  limitations,  the  focus  of  this  paper  is  on  the  expert 
system  and  the  rational  functioning  of  the  tutor.  Some  results  of 
the  ongoing  research  on  the  emotional  aspects  of  the  interaction 
student-tutor,  such  as  recognition  of  students’  emotions  and 
expression  of  emotions  by  the  tutor  can  be  found  at  (Jaques,  Leh¬ 
mann,  &  Pesty,  2009;  Jaques  et  al.,  2011;  Motola,  Jaques,  8i  Axt, 
2008). 

In  the  next  section,  we  describe  PATequation.  It  is  an  editor  the 
student  can  use  to  solve  equations  step-by-step  with  immediate 
feedback.  It  implements  the  inner  loop  of  PAT2Math.  Currently, 
PATequation  assigns  tasks  (outer  loop)  in  a  fixed  sequence  previ¬ 
ously  defined  by  a  professor. 

6.  PATequation 

PATequation  is  a  Java  applet  that  assists  students  in  solving  lin¬ 
ear  and  quadratic  equations  in  PAT2Math.  Fig.  1  illustrates  the 
interface  of  PATequation. 

PATequation  is  comprised  of  an  Editor  (Fig.  1)  where  students 
exercise  problem  solving  and  an  Expert  System  that  provides 
step-by-step  guidance  for  students.  In  Fig.  1,  on  the  green  black¬ 
board,  the  first  line  (labeled  A)  contains  the  equation  the  tutor  se¬ 
lected  for  the  student  to  solve.  The  next  lines  represent  step-by- 
step  solutions  provided  by  the  students.  The  tutor  offers  immedi¬ 
ate  feedback  for  each  step,  indicating  whether  or  not  a  response 
is  correct  by  using  thumbs  up  and  thumbs  down  symbols.  For  each 
step,  the  student  should  choose  an  operation  to  apply  and  also  pro¬ 
vide  the  resulting  equation. 

The  student  chooses  an  operation  by  clicking  a  button  with  the 
corresponding  symbol  from  one  of  the  right  panels.  The  selected 
operation  symbol  will  appear  in  the  step  line  on  the  left  side  of 
the  equation  (see  letter  D  in  Fig.  1).  This  allows  the  tutor  to  evalu¬ 
ate  two  things:  (1)  whether  the  students  knew  which  operation  to 
use  to  solve  the  problem  and  (2)  whether  they  applied  the  opera¬ 
tion  correctly.  It  is  not  possible  to  evaluate  the  former  skill  if  the 


Fig.  1.  PATequation  interface.  PATequation  is  a  Java  applet  that  assists  students  when  solving  linear  and  quadratic  equations  in  PAT2Math.  It  provides  the  student  with  step- 
by-step  guidance. 
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tutor  interface  does  not  ask  students  to  explicitly  choose  an  oper¬ 
ation  to  apply.  To  allow  advanced  learners  to  operate  freely,  PAT- 
equation  permits  the  student  to  provide  a  step  solution  that  uses 
several  operations.  For  example,  for  the  equation  3x  +  5  =  2x  -  7 
(see  Fig.  1 ),  a  student  may  provide  as  the  next  step  x  =  -7  -  5  in¬ 
stead  of  creating  three  steps:  3x  -  2x  +  5  =  -7,  3x-2x  =  -7  -  5, 
x  =  -7  -  5.  In  this  case,  the  system  allows  the  student  to  choose 
anyone  of  the  operations  applied  in  the  three  steps.  The  tutor  noti¬ 
fies  students  when  they  reach  the  final  solution,  i.e.,  when  they 
find  the  value  of  the  variable. 

At  any  time  during  the  problem-solving  process,  the  students 
can  also  choose  one  of  the  following  assistance  options:  (1)  They 
can  ask  for  a  hint,  (2)  They  can  request  that  the  tutor  solve  the  next 
step  of  the  equation,  or  (3)  They  can  request  that  the  tutor  com¬ 
plete  the  equation  from  the  current  step. 

PAT2Math  has  all  of  the  functionalities  described  above  (imme¬ 
diate  feedback,  hint,  solve  the  next  step  and  solve  the  full  prob¬ 
lem).  In  the  next  section,  we  describe  in  more  detail  the 
implementation  of  this  expert  system. 


7.  PATequation  expert  system 

In  order  to  provide  immediate  feedback  for  student  steps  during 
the  problem  solving  process,  PATequation  must  be  able  to  solve 
linear  and  quadratic  equations.  Additionally  it  must  be  able  to  ver¬ 
ify  that  the  partial  solution  provided  by  the  student  matches  one  of 
the  equations  obtained  by  the  tutor.  Although  PATequation  has 
both  of  these  features,  the  features  represent  different  modules 
of  an  ITS  and  involve  different  types  of  rules.  The  former  is  called 
the  step  generator  because  it  generates  the  next  step  for  the  student 
(Vanlehn,  2006).  It  also  provides  the  next-step  hint.  The  latter  is 
called  the  step  analyzer  because  it  compares  the  step  provided  by 
the  student  with  all  possible  steps  found  by  the  step  generator.  It 
then  classifies  the  student’s  solution  as  either  “correct”  or  “incor¬ 
rect”  (Vanlehn,  2006).  It  is  this  combination  of  the  step  generator 
and  the  step  analyzer  that  we  refer  to  as  the  “expert  system”. 
When  it  is  necessary  to  make  explicit  which  set  of  rules  we  are 
mentioning,  we  refer  to  them  as  Step  Generator  (SG)  or  Step  Ana¬ 
lyzer  (SA),  respectively.  Fig.  2  represents  the  architecture  of  PAT2- 
Math  Expert  System. 

PATequation  uses  53  rules  for  the  SG  and  21  rules  for  the  SA.  In 
the  SG,  each  rule  generally  represents  an  algebraic  operation  to 
solve  a  linear  or  quadratic  equation.  They  are  knowledge  compo¬ 
nents  in  the  Algebra  domain.  For  example,  in  order  to  solve  linear 
equations,  students  can  use  one  of  the  following  operations:  basic 
arithmetic  operations  (add,  subtract,  multiply  or  divide)  on  real 
numbers  or  on  variables;  a  transformation  (add,  subtract,  multiply 
or  divide  both  sides  of  the  equation):  simplification  of  fraction,  or 
least  common  multiple.  In  order  to  solve  quadratic  equations,  they 
can  apply  the  following  operations  in  addition  to  the  previous 
ones:  factorization,  expansion,  quadratic  formula,  square  of  the 
sum  of  two  numbers,  difference  of  the  square  formula,  square  of 
the  difference,  rationalization,  potentiation,  and  square  root.  Of 
the  53  rules  available  in  the  SG,  20  rules  aim  at  replicating  stu¬ 
dents’  misconceptions  (described  bellow  in  more  detail).  The 
remaining  rules  are  used  to  rewrite  the  equations  (for  instance, 
combine  like  terms)  and  other  complementary  operations  (find 
the  final  roots  for  quadratic  equations). 

The  expert  system  is  invoked  by  PATequation’s  graphical  inter¬ 
face  when  students  request  a  demonstration  of  how  to  solve  an 
equation  (by  clicking  either  the  “Show  Step"  button  or  the  “Show 
Resolution”  button  shown  in  Fig.  1 )  or  when  they  want  to  verily 
if  the  step  they  provided  is  correct.  When  a  student  asks  the  tutor 
to  show  the  equation  solution,  the  ES  calls  the  SG,  which  receives 
the  equation  and  solves  it  from  the  current  step.  To  verify  if  the 
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Fig.  2.  The  Architecture  of  PAT2Math  Expert  System.  PAT2Math’s  expert  system 
comprises  two  modules  and  three  rules  databases.  The  step  generator  generates  the 
next  step  of  the  solution.  The  step  analyzer  compares  the  step  provided  by  the 
student  with  all  the  possible  solutions  found  by  the  step  generator.  It  then  classifies 
the  student’s  solution  as  either  “correct"  or  “incorrect". 


student  step  is  correct,  the  ES  receives  the  equation  along  with 
the  student’s  current  step  and  sends  them  to  the  SA.  The  SA  re¬ 
quests  that  the  SG  solve  the  equation  and  then  verifies  whether 
the  SG  is  able  to  calculate  the  same  step  provided  by  the  student. 
If  it  cannot,  the  SA  requests  that  the  SG  solve  the  equation  using 
the  rules  that  represent  students'  misconception.  If  it  does  not  rec¬ 
ognize  the  step  provided  by  the  student,  the  SA  gives  the  student  a 
hint.  Otherwise,  the  SA  simply  indicates  that  the  step  is  incorrect. 


7.1.  Representation  format  for  equations 

In  order  for  the  ES  to  solve  equations,  it  is  necessary  that  the 
data  structure  to  represent  equations  is  easy  to  operate  on  and  is 
unambiguous.  For  example,  consider  the  equation  2x  +  3  *  x  =  6.  It 
is  ambiguous  unless  one  knows  that  multiplication  happens  before 
subtraction.  A  natural  data  structure  for  representing  an  equation 
is  a  binary  tree  because  it  defines  the  order  in  which  operations 
happen.  For  this  reason,  compilers  have  largely  used  it  to  represent 
mathematical  expressions  (Aho,  2007).  A  binary  tree  that  repre¬ 
sents  an  equation  is  called  an  “expression  tree”.  Fig.  3  shows  exam¬ 
ples  of  expression  trees  for  the  equations  2x  +  3x  =  6  and 
(2x  +  3)  *  x  =  6,  respectively. 

There  are  three  types  of  nodes  in  an  expression  tree: 

-  constant  nodes :  hold  real  or  integer  numbers, 

-  identifier  nodes:  denote  variables, 

-  compound  nodes:  represent  the  application  of  a  operator  to  two 

operands,  each  one  being  another  expression  tree. 

As  Fig.  3  illustrates,  the  constant  nodes  and  the  identifier  nodes 
are  always  leaf  nodes  (a  node  that  has  no  child).  On  the  other  hand, 
the  compound  nodes  have  either  one  (unary  operation)  or  two 
(binary  operation)  children  trees,  which  represent  the  operands 
of  the  operation. 


data  structure  that  PAT2Math  ES  uses  to  represent  equations. 


PA.  Jaqi 


:  al/Expert  Systems  with  Applications  40  (2013)  5456-5465 


5461 


Fig.  4.  An  example  of  ES  reasoning  cycle  based  on  (Hopgood,  2000).  For  every  cycle,  the  ES  selects  from  the  knowledge  base  those  rules  that  match  with  facts  in  the  working 
memory  and  stores  them  in  the  Agenda.  It  triggers  a  rule  from  the  Agenda  according  to  a  determined  selection  strategy.  It  also  empties  the  Agenda  and  starts  a  new  cycle  until 
the  Agenda  has  no  more  rules  to  trigger. 


7.2.  PATequation  step  generator 

In  a  rule-based  expert  system,  the  inference  engine  (IE)  exe¬ 
cutes  several  cycles  of  reasoning  to  solve  problems.  In  each  cycle, 
it  executes  the  following  actions  (shown  in  Fig.  4).  First,  it  selects, 
from  the  knowledge  base,  the  rules  that  correspond  to  facts  in  the 
working  memory  and  stores  them  in  the  Agenda,  a  special  memory 
area  that  stores  that  rules  selected  to  be  triggered.  Among  these 
rules,  it  chooses  one  rule  to  trigger  according  to  a  determined  strat¬ 
egy  of  selection.  For  example,  in  PATequation,  each  rule  has  a  pri¬ 
ority  number  and  the  IE  selects  the  rule  with  highest  priority  to 
trigger.  In  the  case  where  all  the  rules  have  the  same  priority, 
the  IE  triggers  the  rule  that  has  matched  those  facts  most  recently 
added  to  the  database.  If  the  rule  produces  new  knowledge,  the  IE 
adds  this  new  fact  to  the  working  memory.  It  also  empties  the 
Agenda  and  starts  a  new  cycle  until  the  Agenda  has  no  more  rules 
left  to  trigger. 

There  are  two  reasoning  mechanisms  in  the  RBES  inference  en¬ 
gine:  Forward  Chaining  (FC)  and  Backward  Chaining  (BC).  Forward 
Chaining  selects  rules  whose  antecedents  match  facts  in  the  work¬ 
ing  memory  while  Backward  Chaining  examines  the  rules  conse¬ 
quents.  While  the  FC  is  more  appropriate  for  diagnostic 
problems,  the  BC  is  more  suitable  for  proving  assertions.  BC  is  more 
efficient  because  it  only  triggers  the  rules  that  are  necessary  to 
solve  the  problem. 

As  previously  explained,  the  PATequation  ES  was  implemented 
with  the  Drools  shell.  The  Drools  IE  applies  Forward  Chaining.  This 
is  the  most  appropriate  type  of  reasoning  for  algebra  problem  solv¬ 
ing  because  the  solution  is  previously  unknown. 

Indeed,  there  are  several  paths  to  solve  an  equation.  Fig.  5  illus¬ 
trates  the  available  paths  to  solve  the  equation  4x  +  4  =  9  -  x.  It  is 
represented  as  a  directed  graph,  in  which  the  nodes  denote  a  step 
solution.  Each  edge  represents  the  operation  that  is  applied  to  the 


equation  in  the  predecessor  node  in  order  to  obtain  the  equation  in 
the  successor  node. 

Even  when  solving  a  very  simple  equation,  such  as  the  afore¬ 
mentioned  equation  in  Fig.  5,  there  are  two  possible  paths.1  When 
demonstrating  how  to  solve  an  equation,  PATequation  should  be 
able  to  choose  a  solution  path.  This  is  implemented  by  applying 
higher  priority  to  the  rules  that  represent  operations  with  greater 
significance.  An  operation  can  have  high  significance  because  it  has 
higher  precedence  or  because  its  application  is  more  desirable  for 
pedagogical  reasons.  For  example,  in  Fig.  5,  although  both  paths 
are  correct,  the  left  path  is  the  more  desirable  because  teachers  gen¬ 
erally  prefer  to  first  isolate  the  variable  on  the  left  of  the  equation  in 
order  to  emphasize  it. 

In  the  forward  chaining  reasoning,  the  first  step  of  the  IE  is 
selecting  the  rules  for  examination.  Generally,  it  selects  all  the 
rules  of  the  KB.  It  evaluates  the  condition  part  of  each  rule.  If  the 
condition  component  is  satisfied,  it  adds  the  rule  to  conflict  set. 
After  examining  all  the  rules,  it  selects  one  conflict  set  rule  to  trig¬ 
ger  according  to  a  conflict  resolution  strategy.  For  example,  the 
strategy  could  select  the  rule  with  higher  priority.  Finally,  the  IE 
updates  the  working  memory  with  the  new  facts  generated  by 
the  rule  and  empties  the  conflict  set  in  order  to  start  a  new  cycle. 

The  following  scenario,  illustrated  in  Table  2,  shows  how  PAT¬ 
equation  solves  the  euation  4x  +  4  =  9  -  x  through  forward  chain¬ 
ing  reasoning.  First,  it  translates  the  equation  from  the  infix  math 
notation  to  postfix  (Goodrich  &  Tamassia,  2010)  in  order  to  convert 
it  into  an  expression  tree  (Lafore,  2003).  It  includes  the  expression 
tree  as  a  new  fact  into  the  working  memory  of  Drools  and  launches 
the  inference  engine.  At  this  moment,  the  FC  reasoning  begins. 
Generally  every  cycle  solves  one  step  of  the  equation. 

In  the  first  cycle,  the  IE  examines  all  rules  in  the  KB  in  order  to 


1  Actually,  there  are  other  paths  if  we  allow  recursive  operations. 
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Fig.  5.  Paths  to  solve  the  equation  4x  +  4  =  9  -x.  There  can  be  several  paths  to 
solving  an  equation.  A  path  is  a  sequence  of  steps  with  each  step  resulting  in  an 
intermediate  equation.  A  step  is  a  user  interface  action  (for  example,  to  simplify  the 
fraction  5x/5  in  an  intermediate  equation)  that  the  student  takes  in  order  to  achieve 
a  task2.  (In  fact,  Vanlehn  differs  step  and  learning  events,  where  the  step  is  a  user 
interface  action  and  the  learning  event  is  the  application  of  a  knowledge  component 
to  achieve  a  task;  it  is  just  a  mental  event.  For  the  sake  of  simplicity,  we  just  refer  to 
steps  in  this  article  to  mean  that  it  is  a  user  action  that  involves  one  or  more 
learning  events.)  This  figure  illustrates  two  possible  paths  to  solve  the  equation 
4x  +  4  =  9  -  x.  The  left  and  right  paths  have  both  six  steps  (the  two  last  steps  are 
equal). 


select  those  rules  whose  antecedents  contain  conditions  that  are 
satisfied  by  the  current  equation.  Because  the  operation  of  addition 
is  only  applied  on  the  same  side  of  the  equation,  this  rule  is  not  eli¬ 
gible.  The  IE  chooses  the  rules  MOVEJNT  and  MOVE.VARIABLE, 
which  aim  at  moving  the  operand  from  one  side  of  the  equation 
to  the  other  by  applying  an  operation  on  both  sides.  While  the  goal 
of  the  MOVEJNT  rule  is  to  move  constants  to  the  right  side,  the 
purpose  of  MOVE_VARIABLE  rule  is  to  move  all  variables  to  the  left 
side.  Both  rules  aim  at  isolating  the  variables  on  the  left  and  con¬ 
stants  on  the  right  side.  The  IE  selects  and  triggers  the  MOVE_VAR- 
IABLE  rule  because  it  has  a  higher  priority.  The  resulting  equation 
( 4x  +  4  +  x  =  9  —x  +  x )  replaces  the  previous  one  in  the  Working 
Memory.  This  prevents  the  IE  from  applying  the  operations 
recursively. 

In  the  second  cycle,  there  are  two  possible  operations  that  the  IE 
can  apply:  (1)  sum  the  variables  on  the  left  and  right  sides  of  the 
equation  and  (2)  apply  MOVEJNT.  MOVE.VARIABLE  is  not  se¬ 
lected  because  it  does  not  satisfy  the  condition  that  the  equation 
only  present  one  variable  term  on  its  right  side.  The  IE  selects 
the  rules  representing  these  two  operations  and  inserts  them  into 


the  Agenda.  The  former  rule  has  higher  priority  and  is  triggered. 
The  resulting  equation  (5x  +  4  =  9)  is  now  the  only  fact  in  the  work¬ 
ing  memory.  In  fact,  the  IE  applies  this  rule  twice.  In  the  second  cy¬ 
cle,  it  adds  4x  +  x  on  the  left  side  of  the  equation.  In  the  third  cycle, 
it  adds  -x  +  x  on  the  right  side. 

In  the  fourth  cycle,  there  is  only  one  rule  eligible  to  be  triggered, 
the  MOVEJNT  rule.  The  IE  triggers  this  rule  and  a  new  equation 
(5x  +  4-  4  =  9- 4)  replaces  the  previous  one  in  the  Working 
Memory. 

In  the  fifth  and  sixth  cycles,  it  selects  and  triggers  the  rule 
ADDJZONSTANTS  twice.  Each  cycle  produces  the  equations 
5x  =  9  -  4  and  5x  =  5,  respectively. 

In  the  next  cycle  (7th),  there  is  only  one  rule  whose  conditions 
match  with  the  fact  5x  =  x  in  the  working  memory.  This  rule  di¬ 
vides  both  sides  of  the  equations  by  5  in  order  to  cancel  the  coef¬ 
ficient  of  the  term  5x.  The  resulting  fact  is  5x/5  =  5/5. 

In  the  following  two  cycles,  the  IE  applies  the  rules  to  simplify 
the  fractions  on  both  side  of  the  equation.  The  result  is  that  the 
equation  x  =  1  is  the  only  fact  left  in  the  Working  Memory. 

In  the  last  cycle,  the  IE  triggers  the  rule  whose  purpose  is  to  test 
if  the  final  response  was  achieved,  i.e.,  PATequation  found  the  final 
value  of  the  equation  variable.  In  this  case,  it  removes  all  the  facts 
of  the  Working  Memory.  This  stops  the  execution  of  the  IE. 

Besides  the  cycles  above  cited,  there  are  other  cycles  in  which 
the  IE  applies  auxiliary  rules  aimed  at  preparing  the  equation  for 
the  next  operation.  For  example,  there  is  a  rule  based  on  the  com¬ 
mutative  property  of  numbers  that  aims  at  grouping  the  similar 
terms  in  the  equation.  It  transforms  the  equation  x  +  4  +  3x  =  0  into 
the  equation  x  +  3x  +  4  =  0.  As  the  objective  of  this  type  of  rule  is 
simply  to  prepare  the  expression  tree  for  an  operation,  we  do  not 
cite  it  in  the  above  scenario  because  we  want  to  focus  on  the  rules 
that  actually  represent  a  real  step  towards  solving  the  problem. 

In  this  section,  we  demonstrated  how  PATEquation  ES  solves  an 
equation.  This  functionality  allows  the  system  to  show  students 
how  to  solve  a  step  or  the  full  equation  from  any  point  of  the  solv¬ 
ing  process  when  they  assistance.  If  the  students  ask  only  for  the 
next  step,  only  one  cycle  of  the  forward  reasoning  is  performed. 
If  the  student  asks  for  the  full  solution,  the  IE  iterates  through  as 
many  cycles  as  necessary  to  solve  the  equation.  It  also  shows  all 
the  facts  generated  in  each  solving  cycle.  In  both  cases,  the  initial 
fact  is  the  current  equation. 

In  the  next  Section,  we  show  how  the  Step  Analyzer  verifies  stu¬ 
dents’  steps  in  order  to  provide  immediate  feedback. 

7.3.  PATequation  step  analyser 

As  we  previously  mentioned,  in  order  for  a  learning  system  to 
be  considered  an  intelligent  tutoring  system  it  must  be  able  to  pro¬ 
vide  immediate  feedback  to  the  students  in  the  inner  loop  (Van- 


of  the  working  memory  and  agenda  for  solving  equation  4x  +  4  =  9  -  x. 

Working  memory  (facts  before  firing  rule)  Working  memory  (facts  after  firing  rule) 

4x  +  4  =  9  —  x  4x  +  4  +  x  =  9-x  +  x 


2  4x  +  4  +  x  =  9  —  x  +  x 

3  5x +  4  =  9  —  x  +  x 


4  5x  +  4  =  9 

5  5x  +  4  —  4  =  9  —  4 

6  5x  =  9  —  4 

7  5x  =  5 

8  5x15  =  5/5 

9  x  =  5/5 

10  x  =  1 


5x  +  4  —  4  =  9  —  4 
5x  =  9  —  4 
5x  =  5 
5x/5  =  5/5 
x  =  5/5 

(EMPTY) 


Agenda  (rules) 

(1)  MOVEJNT 

(2)  MOVE_VARIABLE 

(1 )  ADD.VARIABLES 

(2)  MOVEJNT 

(1 )  ADD.VARIABLES 

(2)  MOVEJNT 
(1)  MOVEJNT 

(1)  ADD.CONSTANTS 
(1)  ADD_CONSTANTS 
(1)  DIVIDE J30TFLSIDES 
(1)  SIMPLIFY JTtACTION 
(1)  SIMPLIFY JTtACTION 
(1)  TEST.END 


Triggered  rules 


MOVE_VARIABLE 
ADD.  VARIABLES 
ADD.  VARIABLES 

MOVEJNT 

ADD_CONSTANTS 

ADD_CONSTANTS 

DIVIDE.BOTH.SIDES 

SIMPLIFY JTtACTION 

SIMPLIFY JTtACTION 

TEST_END 
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(1)  The  SA  executes  the  SG  in  order  to  obtain  the  next  step  of  the 
equation; 

(2)  If  there  is  no  other  possible  solution,  it  returns  the  linked  list; 

(3)  If  there  is  a  solution: 

linked  list; 

(b)  It  compares  the  student  equation  (stdEq)  and  the  resulting 
equation: 


(i)  If  they  are  equivalent,  it  returns  the  linked  list. 

(ii)  Otherwise,  it  recursively  calls  the  current  function 
passing  the  resulting  equation  as  the  curEq  parameter. 


Fig.  6.  PATequation  step  analyser  algorithm.  The  SA  verifies  students’  steps  in  order  to  provide  immediate  feedback. 


lehn,  2006).  Additionally  it  should  be  able  to  verify  which  opera- 
tions  they  used  correctly  and  which  they  used  incorrectly.  The  Step 
Analyzer  (SA)  accomplishes  this  task,  which  is  also  called  Model 
Tracing  (Anderson,  Corbett,  Koedinger,  &  Pelletier,  1995;  Heffernan 
et  al.,  2008). 

The  SA  is  a  recursive  function  that  receives  as  parameter  the 
current  equation  (curEq)  and  the  student  step  (stdEq)  and  returns 
a  linked  list  containing  the  sequences  of  operations  necessary  to 
achieve  the  student’s  answer.  The  pseudo-code  below  demon¬ 
strates  how  this  recursive  function  works. 

The  recursive  algorithm  in  Fig.  6  allows  backtracking,  i.e.,  the  IE 
can  return  to  a  previous  state  in  order  to  try  another  solution  path. 
While  backtracking  is  a  powerful  process,  it  can  significantly  de¬ 
crease  the  system  performance.  Without  using  any  heuristic  to 
minimize  the  possible  paths,  the  FC  performs  a  full  breadth-first 
search,  resulting  in  a  worst  case  runtime  complexity  of  0(nd), 
where  n  is  the  maximum  branching  factor  of  the  search  tree  (in 
an  ES  the  number  of  possible  rules)  and  d  is  the  depth  of  the 
last-cost  solution  (number  of  triggered  rules)  (Russell  &  Norvig, 
2003).  In  PATequation,  for  the  worst  cases,  the  value  of  n  can  be 
53,  which  are  the  numbers  of  rules  available  in  the  SG  set. 

In  order  to  minimize  the  number  of  unnecessary  triggered  rules, 
we  create  meta-rules  that  infer  the  operation  that  the  student  ap¬ 
plied.  These  additional  rules  compare  the  current  equation  and  the 
student  step  in  order  to  identify  which  operations  the  student  most 
likely  applied.  For  example,  let  us  assume  that  the  current  equation 
is  1  +  2  +  3  =  x  and  the  student  step  response  is  1  +  5  =  x.  The  meta¬ 
rules  verify  that  the  terms  1  and  x  are  still  present  in  the  student 
equation,  but  not  the  terms  2  and  3.  They  also  verify  that  there  is 
an  add  operator  between  these  terms.  In  that  case,  it  tries  to  apply 
the  rule  ADD_CONSTANTS  for  these  two  operands  in  order  to  try  to 
find  5.  With  the  meta-rules,  the  time  complexity  is  reduced  to  O 
(d),  where  d  is  the  number  of  triggered  rules,  it  means,  the  number 
of  rules  that  need  to  be  triggered  in  order  to  achieve  the  solution.  It 
happens  because  the  meta-rules  guide  the  FC  to  trigger  only  the 
rules  that  are  necessary  to  find  the  student’s  solution. 

In  fact,  these  meta-rules  try  to  emulate  a  backward  chaining  in 
the  expert  system,  which  is  the  most  appropriate  reasoning  for  this 
type  of  problem.  Backward  reasoning  only  triggers  the  rules  that 
are  necessary  to  solve  the  problem.  However,  Drools  and  other  sta¬ 
ble  expert  systems  shells  do  not  implement  this  type  of  reasoning. 
This  is  the  reason  why  we  decided  to  implement  this  meta-rules 
approach. 

Another  potential  problem  with  expert  systems  is  the  number 
of  comparisons  that  need  to  be  made  between  rules  and  facts  in 
the  database.  Most  current  ES  shells  use  a  RETE  algorithm  for  this 
match  process,  which  reduces  the  average  complexity  from  O  (nd) 
to  0  (n2),  where  n  is  the  number  of  facts  in  the  Working  Memory 
(Watson,  2008).  As  we  generally  work  with  few  facts  (generally 
two)  in  PATequation,  this  process  does  not  impair  its  performance. 


8.  Evaluation 

In  order  to  evaluate  the  effectiveness  of  our  tool,  we  compared 
the  students’  improvement  in  performance  in  a  group  of  22 
students  who  solved  equations  in  PATequation  (experimental 
group-EG)  in  relation  to  a  group  of  21  students  who  only  exercised 
without  the  tool  (control  group-CG). 

The  hypothesis  of  the  experiment  is  that  PATequation  helps  the 
student  in  learning  linear  equations  providing  a  statistically  higher 
gain  scores  (difference  between  posttests  and  pretests)  compared 
to  students  who  exercised  equation  without  the  tool.  The  null 
and  alternative  hypotheses  are: 

H0:  EGgain  ^  CGgain  (null  hypothesis) 

H,:  EGgain  >  CGgain  (alternative  hypothesis) 

The  null  hypothesis  is  that  the  gain  scores  in  the  EG  is  less  than 
or  equal  to  the  CG  gain.  Moreover,  the  claim  is  that  the  EG  gain  was 
statistically  higher  than  the  CG  gain,  identifying  that  students  exer¬ 
cising  with  PATequation  learned  more  than  students  who  only 
solved  equations  in  paper-and-pencil. 

The  experimental  evaluation  took  place  in  June  2012  in  a  pri¬ 
vate  school  in  Porto  Alegre,  Brazil.  Forty-three  students  from  two 
seventh  grade  classes  of  an  elementary  school  participated  in  the 
experiment.  Because  the  students  were  minors  (12-13  years),  we 
asked  the  parents  to  sign  the  informed  consent  form  before  con¬ 
ducting  our  study.  This  evaluation  can  be  classified  as  a  quasi¬ 
experiment,  because  the  sample  subjects  were  not  chosen  ran¬ 
domly  (Campbell  &  Stanley,  1963).  These  classes  were  selected  be¬ 
cause  of  their  previous  contact  with  our  research  group  and  also 
because  of  the  interest  of  their  teachers. 

Fig.  7  illustrates  the  phases  of  the  evaluation  experiment.  Ini¬ 
tially,  the  subjects  were  informed  about  the  objectives  of  the  exper¬ 
iment  (evaluate  PATequation),  as  well  as  the  voluntary  and 
confidential  nature  of  their  participation.  The  experiment  was  con¬ 
ducted  in  4  sessions  each  lasting  for  50  min,  totaling  3  h  and  20  min 
of  interaction.  In  the  first  lesson,  we  administered  a  pretest  aimed  at 
identifying  the  students’  prior  knowledge.  During  the  following  two 
classes,  the  students  in  the  experimental  group  solved  equations 
with  our  tool  and  students  in  the  control  group  used  only  paper 
and  pencil.  In  the  last  class  they  solved  the  posttest  and  completed 
the  questionnaire.  The  questionnaire  consisted  of  four  questions 
regarding  students’  habits  when  studying  math  and  thirteen  ques¬ 
tions  about  the  students’  experience  with  PATequation  after  having 
used  it  (Fig.  8  shows  the  students’  answers  for  one  of  the  questions). 

The  students’  grades  and  the  mean  and  standard  deviation  of 
pretest  and  posttest  in  control  and  experimental  groups  are  shown 
in  Table  3.  Each  test  was  composed  of  5  problems  for  students  to 
solve,  worth  a  total  of  2.5  points.  The  equations  in  each  test  were 
different,  but  evaluated  the  same  skills.  Table  4  shows  the  mean 
and  standard  deviation  for  pretests  and  posttests  scores. 
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.  I-  I  J  Paper-pencil  I _ J  Posttest  I - >1  Questionnaire  I 

(^1 - H  Pretest  I  H  exercises  J  1 - 1  1 - 1 


Experimental  Group 


Q - >|  Pretest  | - »[  PATequation] - >1  Posttest  I - H  Questionnaire^ 


Fig.  7.  The  PAT2Math  evaluation  experiment.  The  experiment  consisted  of  3  phases:  (1 )  pretest,  (2)  solving  equations  in  paper-and-pencil  for  I 
equations  in  PAT2Math  for  the  experimental  group,  and  (3)  posttest  and  questionnaire. 


I  group,  or  solving 


5% 

■  Very  interesting 

■  Interesting 

■  Good 

■  Boring 

■  Very  boring 

Fig.  8.  Students'  answers  to  the  question  “Solving  equations  in  PATequation  was 
. . After  completing  the  posttest,  students  were  asked  to  complete  a  questionnaire 
regarding  their  math  habits  and  their  experience  with  PATequation. 


posttest  mean , 


deviation  (sd). 


Pretest  Posttest  Gain 


Control  group 

sd  0.64 

Experimental  group 
Mean  1.6 

sd  0.59 


1.86 

0.59 


2.2 

0.29 


Student  Pretest  Posttest  Gain 


Experimental  Group 


Pretest  Posttest  Gain 


We  used  a  one-tail  t-test  because  we  did  not  know  the  popula¬ 
tion  standard  deviation  (we  only  know  the  sample  s).  With  a  con¬ 
fidence  level  of  90%  (a  =  0.1),  we  obtain  p  =  0.06  (t= -1.5376, 
df  =  39).  The  mean  score  increased  from  1.445  (sd  =  0.642)  on  the 
pretest  to  1.865  (sd  =  0.599)  on  the  posttest.  The  difference  be¬ 
tween  the  two  means  is  statistically  significant  at  the  .1  level.  Thus, 
as  p  <  a,  we  reject  the  null  hypothesis  and  conclude  that  there  is 
evidence  to  say  with  90%  confidence  that  students  using  PATequa¬ 
tion  have  higher  gain  scores  in  equation  solving  than  students 
using  pen  and  pencil. 

On  the  questionnaire,  22%  of  students  reported  it  was  very 
interesting  to  solve  equations  in  PATEquation,  56%  of  them  stated 
it  was  interesting  and  for  22%  of  students,  it  was  good.  Finally,  52% 


of  students  agreed  that  solving  equations  in  PATequation  with 
the  help  of  hints  increased  their  knowledge  about  algebraic 
equations. 

9.  Conclusions 

In  this  paper  we  presented  PAT2Math,  an  expert  system  module 
of  an  Algebra  ITS.  The  ES  is  responsible  for  correcting  student  steps 
and  modeling  student  knowledge  components  during  problem 
solving  in  the  PAT2Math’s  editor  (PATequation).  Another  impor¬ 
tant  function  of  this  module  is  to  demonstrate  to  the  student 
how  to  solve  a  problem.  In  order  to  provide  this  step-by-step  guid¬ 
ance  in  the  inner  loop,  ES  is  able  to  solve  the  same  problems  that 
student  do. 

The  proposed  ES  was  implemented  as  rule-based  expert  system 
in  which  one  rule  is  used  to  represent  each  algebra  operation  a  stu¬ 
dent  should  master.  The  ES  is  composed  by  meta-rules  that  aim  at 
reducing  the  number  of  possible  paths  to  explore  when  correcting 
a  student’s  solution.  This  solution  reduces  the  computational  com¬ 
plexity  from  0(nd)  to  O(d),  where  n  is  the  number  of  rules  in  the 
knowledge  base  and  d  is  the  number  of  rules  triggered. 

In  this  paper,  we  mainly  focused  on  the  expert  system  of  PAT2- 
Math,  describing  its  implementation  and  its  approach  on  solving 
equations  and  providing  feedback  to  the  user.  We  also  described 
a  quasi-experiment  with  forty-three  7th  grade  students,  which 
demonstrated  that  the  performance  improvement  observed  in  stu¬ 
dents  using  PATequation  to  solve  equations  was  statistically  signif¬ 
icant.  Encouraged  by  our  initial  results  we  plan  to  conduct  a  new 
user  study  involving  more  students  over  a  longer  period  of  time. 
We  expect  to  gain  new  insights  about  the  long-term  applicability 
and  scalability  of  our  approach. 
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